Face Generation¶

In this project, you'll use generative adversarial networks to generate new images of faces.

Get the Data¶

You'll be using two datasets in this project:

MNIST
CelebA

Since the celebA dataset is complex and you're doing GANs in a project for the first time, we want you to test your neural network on MNIST before CelebA. Running the GANs on MNIST will allow you to see how well your model trains sooner.

If you're using FloydHub, set data_dir to "/input" and use the FloydHub data ID "R5KrjnANiKVhLWAkpXhNBe".

#data_dir = './data'

# FloydHub - Use with data ID "R5KrjnANiKVhLWAkpXhNBe"
data_dir = '/input'


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import helper

helper.download_extract('mnist', data_dir)
helper.download_extract('celeba', data_dir)

Found mnist Data
Found celeba Data

Explore the Data¶

MNIST¶

As you're aware, the MNIST dataset contains images of handwritten digits. You can view the first number of examples by changing show_n_images.

show_n_images = 25

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
%matplotlib inline
import os
from glob import glob
from matplotlib import pyplot

mnist_images = helper.get_batch(glob(os.path.join(data_dir, 'mnist/*.jpg'))[:show_n_images], 28, 28, 'L')
pyplot.imshow(helper.images_square_grid(mnist_images, 'L'), cmap='gray')

<matplotlib.image.AxesImage at 0x7f865c56c080>

CelebA¶

The CelebFaces Attributes Dataset (CelebA) dataset contains over 200,000 celebrity images with annotations. Since you're going to be generating faces, you won't need the annotations. You can view the first number of examples by changing show_n_images.

show_n_images = 25

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
mnist_images = helper.get_batch(glob(os.path.join(data_dir, 'img_align_celeba/*.jpg'))[:show_n_images], 28, 28, 'RGB')
pyplot.imshow(helper.images_square_grid(mnist_images, 'RGB'))

<matplotlib.image.AxesImage at 0x7f865c466128>

Preprocess the Data¶

Since the project's main focus is on building the GANs, we'll preprocess the data for you. The values of the MNIST and CelebA dataset will be in the range of -0.5 to 0.5 of 28x28 dimensional images. The CelebA images will be cropped to remove parts of the image that don't include a face, then resized down to 28x28.

The MNIST images are black and white images with a single color channel while the CelebA images have 3 color channels (RGB color channel).

Build the Neural Network¶

You'll build the components necessary to build a GANs by implementing the following functions below:

model_inputs
discriminator
generator
model_loss
model_opt
train

Check the Version of TensorFlow and Access to GPU¶

This will check to make sure you have the correct version of TensorFlow and access to a GPU

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
from distutils.version import LooseVersion
import warnings
import tensorflow as tf

# Check TensorFlow Version
assert LooseVersion(tf.__version__) >= LooseVersion('1.0'), 'Please use TensorFlow version 1.0 or newer.  You are using {}'.format(tf.__version__)
print('TensorFlow Version: {}'.format(tf.__version__))

# Check for a GPU
if not tf.test.gpu_device_name():
    warnings.warn('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))

TensorFlow Version: 1.1.0
Default GPU Device: /gpu:0

Input¶

Implement the model_inputs function to create TF Placeholders for the Neural Network. It should create the following placeholders:

Real input images placeholder with rank 4 using image_width, image_height, and image_channels.
Z input placeholder with rank 2 using z_dim.
Learning rate placeholder with rank 0.

Return the placeholders in the following the tuple (tensor of real input images, tensor of z data)

import problem_unittests as tests

def model_inputs(image_width, image_height, image_channels, z_dim):
    """
    Create the model inputs
    :param image_width: The input image width
    :param image_height: The input image height
    :param image_channels: The number of image channels
    :param z_dim: The dimension of Z
    :return: Tuple of (tensor of real input images, tensor of z data, learning rate)
    """
    # TODO: Implement Function
    input_real = tf.placeholder(tf.float32, [None, image_width, image_height, image_channels], name='input_real')
    input_z = tf.placeholder(tf.float32, [None, z_dim], name='input_z')
    learning_rate = tf.placeholder(tf.float32, None, name='learning_rate')
    return input_real, input_z, learning_rate


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_inputs(model_inputs)

Tests Passed

Discriminator¶

Implement discriminator to create a discriminator neural network that discriminates on images. This function should be able to reuse the variables in the neural network. Use tf.variable_scope with a scope name of "discriminator" to allow the variables to be reused. The function should return a tuple of (tensor output of the discriminator, tensor logits of the discriminator).

def discriminator(images, reuse=False, alpha=0.2):
    """
    Create the discriminator network
    :param images: Tensor of input image(s)
    :param reuse: Boolean if the weights should be reused
    :return: Tuple of (tensor output of the discriminator, tensor logits of the discriminator)
    """
    # TODO: Implement Function
    with tf.variable_scope('discriminator', reuse=reuse):
        x = tf.layers.conv2d(inputs=images, filters=64, kernel_size=5, strides=2, padding='same')
        x = tf.maximum(alpha * x, x)
        
        def conv_layer(inputs, filters, kernel_size=5, strides=2, padding='same'):
            conv_layer = tf.layers.conv2d(inputs=inputs, filters=filters, kernel_size=kernel_size,
                                          strides=strides, padding=padding)
            conv_layer = tf.layers.batch_normalization(inputs=conv_layer, training=True)
            conv_layer = tf.maximum(alpha * conv_layer, conv_layer)
            conv_layer = tf.nn.dropout(x=conv_layer, keep_prob=0.50)
            return conv_layer
        
        conv_layer1 = conv_layer(inputs=x, filters=128)
        conv_layer2 = conv_layer(inputs=conv_layer1, filters=256)
        conv_layer3 = conv_layer(inputs=conv_layer2, filters=512)
        conv_layer3_volume = conv_layer3.get_shape().as_list()[1] * conv_layer3.get_shape().as_list()[2] * conv_layer3.get_shape().as_list()[3]
        conv_layer3_flattened = tf.reshape(conv_layer3, [-1, conv_layer3_volume])
        logits = tf.layers.dense(inputs=conv_layer3_flattened, units=1)
        out = tf.sigmoid(x=logits)
        
        return out, logits

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_discriminator(discriminator, tf)

Tests Passed

Generator¶

Implement generator to generate an image using z. This function should be able to reuse the variables in the neural network. Use tf.variable_scope with a scope name of "generator" to allow the variables to be reused. The function should return the generated 28 x 28 x out_channel_dim images.

def generator(z, out_channel_dim, is_train=True, alpha=0.2):
    """
    Create the generator network
    :param z: Input z
    :param out_channel_dim: The number of channels in the output image
    :param is_train: Boolean if generator is being used for training
    :return: The tensor output of the generator
    """
    # TODO: Implement Function
    with tf.variable_scope('generator', reuse=not is_train):
        x = tf.layers.dense(inputs=z, units=7*7*1024)
        x = tf.reshape(x, shape=[-1, 7, 7, 1024])
        x = tf.layers.batch_normalization(inputs=x, training=is_train)
        x = tf.maximum(alpha * x, x)

        def conv2d_transpose(inputs, filters, kernel_size=5, strides=2, padding='same', normalize=True):
            conv_layer_transpose = tf.layers.conv2d_transpose(inputs=inputs, filters=filters, kernel_size=kernel_size,
                                                              strides=strides, padding=padding)
            if normalize:
                conv_layer_transpose = tf.layers.batch_normalization(inputs=conv_layer_transpose, training=is_train)
            conv_layer_transpose = tf.maximum(alpha * conv_layer_transpose, conv_layer_transpose)
            
            return conv_layer_transpose

        conv_layer_transpose1 = conv2d_transpose(inputs=x, filters=512)
        conv_layer_transpose2 = conv2d_transpose(inputs=conv_layer_transpose1, filters=256)
        conv_layer_transpose3 = conv2d_transpose(inputs=conv_layer_transpose2, filters=128, strides=(1, 1))

        logits = conv2d_transpose(inputs=conv_layer_transpose3, filters=out_channel_dim, strides=(1, 1), 
                                  normalize=False)
        out = tf.tanh(x=logits) * 0.50
        
        return out

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_generator(generator, tf)

Tests Passed

Loss¶

Implement model_loss to build the GANs for training and calculate the loss. The function should return a tuple of (discriminator loss, generator loss). Use the following functions you implemented:

discriminator(images, reuse=False)
generator(z, out_channel_dim, is_train=True)

def model_loss(input_real, input_z, out_channel_dim):
    """
    Get the loss for the discriminator and generator
    :param input_real: Images from the real dataset
    :param input_z: Z input
    :param out_channel_dim: The number of channels in the output image
    :return: A tuple of (discriminator loss, generator loss)
    """
    # TODO: Implement Function
    g_model = generator(z=input_z, out_channel_dim=out_channel_dim)
    d_model_real, d_logits_real = discriminator(images=input_real)
    d_model_fake, d_logits_fake = discriminator(images=g_model, reuse=True)
    
    g_loss = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_fake, 
                                             labels=tf.ones_like(d_model_fake)))
    d_loss_real = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_real,
                                             labels=tf.ones_like(d_model_real)))
    d_loss_fake = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=d_logits_fake,
                                             labels=tf.zeros_like(d_model_fake)))
    
    d_loss = d_loss_real + d_loss_fake
    
    return d_loss, g_loss


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_loss(model_loss)

Tests Passed

Optimization¶

Implement model_opt to create the optimization operations for the GANs. Use tf.trainable_variables to get all the trainable variables. Filter the variables with names that are in the discriminator and generator scope names. The function should return a tuple of (discriminator training operation, generator training operation).

def model_opt(d_loss, g_loss, learning_rate, beta1):
    """
    Get optimization operations
    :param d_loss: Discriminator loss Tensor
    :param g_loss: Generator loss Tensor
    :param learning_rate: Learning Rate Placeholder
    :param beta1: The exponential decay rate for the 1st moment in the optimizer
    :return: A tuple of (discriminator training operation, generator training operation)
    """
    # TODO: Implement Function
    trainable_vars = tf.trainable_variables()
    d_vars = [var for var in trainable_vars if var.name.startswith('discriminator')]
    g_vars = [var for var in trainable_vars if var.name.startswith('generator')]
    
    with tf.control_dependencies(tf.get_collection(tf.GraphKeys.UPDATE_OPS)):
        d_train_op = tf.train.AdamOptimizer(learning_rate=learning_rate, beta1=beta1).minimize(d_loss, var_list=d_vars)
        g_train_op = tf.train.AdamOptimizer(learning_rate=learning_rate, beta1=beta1).minimize(g_loss, var_list=g_vars)
    
    return d_train_op, g_train_op


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_opt(model_opt, tf)

Tests Passed

Neural Network Training¶

Show Output¶

Use this function to show the current output of the generator during training. It will help you determine how well the GANs is training.

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import numpy as np

def show_generator_output(sess, n_images, input_z, out_channel_dim, image_mode):
    """
    Show example output for the generator
    :param sess: TensorFlow session
    :param n_images: Number of Images to display
    :param input_z: Input Z Tensor
    :param out_channel_dim: The number of channels in the output image
    :param image_mode: The mode to use for images ("RGB" or "L")
    """
    cmap = None if image_mode == 'RGB' else 'gray'
    z_dim = input_z.get_shape().as_list()[-1]
    example_z = np.random.uniform(-1, 1, size=[n_images, z_dim])

    samples = sess.run(
        generator(input_z, out_channel_dim, False),
        feed_dict={input_z: example_z})

    images_grid = helper.images_square_grid(samples, image_mode)
    pyplot.imshow(images_grid, cmap=cmap)
    pyplot.show()

Train¶

Implement train to build and train the GANs. Use the following functions you implemented:

model_inputs(image_width, image_height, image_channels, z_dim)
model_loss(input_real, input_z, out_channel_dim)
model_opt(d_loss, g_loss, learning_rate, beta1)

Use the show_generator_output to show generator output while you train. Running show_generator_output for every batch will drastically increase training time and increase the size of the notebook. It's recommended to print the generator output every 100 batches.

def train(epoch_count, batch_size, z_dim, learning_rate, beta1, get_batches, data_shape, data_image_mode):
    """
    Train the GAN
    :param epoch_count: Number of epochs
    :param batch_size: Batch Size
    :param z_dim: Z dimension
    :param learning_rate: Learning Rate
    :param beta1: The exponential decay rate for the 1st moment in the optimizer
    :param get_batches: Function to get batches
    :param data_shape: Shape of the data
    :param data_image_mode: The image mode to use for images ("RGB" or "L")
    """
    # TODO: Build Model
    image_width = data_shape[1]
    image_height = data_shape[2]
    out_channel_dim = data_shape[3]
    input_real, input_z, learning_rate_p = model_inputs(image_width=image_width, image_height=image_height, 
                                                      image_channels=out_channel_dim, z_dim=z_dim)
    d_loss, g_loss = model_loss(input_real=input_real, input_z=input_z, out_channel_dim=out_channel_dim)
    d_train_opt, g_train_opt = model_opt(d_loss=d_loss, g_loss=g_loss, learning_rate=learning_rate, beta1=beta1) 
    
    step = 0
    
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        for epoch_i in range(epoch_count):
            for batch_images in get_batches(batch_size):
                # TODO: Train Model
                step += 1
                
                batch_z = np.random.uniform(-0.5, 0.5, size=[batch_size, z_dim])
                
                _ = sess.run(d_train_opt, 
                             feed_dict={input_real: batch_images, input_z: batch_z, learning_rate_p: learning_rate})
                _ = sess.run(g_train_opt, feed_dict={input_real: batch_images, input_z: batch_z, 
                                                     learning_rate_p: learning_rate})
                
                train_loss_d = sess.run(d_loss, feed_dict={input_real: batch_images, input_z: batch_z})
                train_loss_g = sess.run(g_loss, feed_dict={input_z: batch_z})
            
                print("Epoch {}/{}".format(epoch_i + 1, epoch_count),
                      "Discriminator loss: {:.4f}".format(train_loss_d),
                      "Generator loss: {:.4f}".format(train_loss_g))
                if step == 1 or step % 20 == 0:
                    show_generator_output(sess=sess, n_images=25, input_z=input_z, 
                                      out_channel_dim=out_channel_dim, image_mode=data_image_mode)

MNIST¶

Test your GANs architecture on MNIST. After 2 epochs, the GANs should be able to generate images that look like handwritten digits. Make sure the loss of the generator is lower than the loss of the discriminator or close to 0.

batch_size = 128
z_dim = 100
learning_rate = .0002
beta1 = .5


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
epochs = 2

mnist_dataset = helper.Dataset('mnist', glob(os.path.join(data_dir, 'mnist/*.jpg')))
with tf.Graph().as_default():
    train(epochs, batch_size, z_dim, learning_rate, beta1, mnist_dataset.get_batches,
          mnist_dataset.shape, mnist_dataset.image_mode)

Epoch 1/2 Discriminator loss: 1.2381 Generator loss: 0.6691

Epoch 1/2 Discriminator loss: 0.2492 Generator loss: 3.1882
Epoch 1/2 Discriminator loss: 0.0661 Generator loss: 5.1176
Epoch 1/2 Discriminator loss: 0.0645 Generator loss: 4.0276
Epoch 1/2 Discriminator loss: 0.0286 Generator loss: 5.0836
Epoch 1/2 Discriminator loss: 0.0108 Generator loss: 6.5405
Epoch 1/2 Discriminator loss: 0.0053 Generator loss: 6.8311
Epoch 1/2 Discriminator loss: 0.0440 Generator loss: 4.1918
Epoch 1/2 Discriminator loss: 0.0444 Generator loss: 4.4503
Epoch 1/2 Discriminator loss: 0.0152 Generator loss: 5.8177
Epoch 1/2 Discriminator loss: 0.0197 Generator loss: 5.9252
Epoch 1/2 Discriminator loss: 0.0301 Generator loss: 5.3568
Epoch 1/2 Discriminator loss: 0.0274 Generator loss: 5.0763
Epoch 1/2 Discriminator loss: 0.0998 Generator loss: 3.8760
Epoch 1/2 Discriminator loss: 0.0506 Generator loss: 7.7917
Epoch 1/2 Discriminator loss: 0.0736 Generator loss: 5.5667
Epoch 1/2 Discriminator loss: 0.1548 Generator loss: 3.2486
Epoch 1/2 Discriminator loss: 0.1977 Generator loss: 10.9933
Epoch 1/2 Discriminator loss: 0.0945 Generator loss: 4.1111
Epoch 1/2 Discriminator loss: 0.9551 Generator loss: 1.1670

Epoch 1/2 Discriminator loss: 1.5131 Generator loss: 16.4468
Epoch 1/2 Discriminator loss: 0.1269 Generator loss: 8.8374
Epoch 1/2 Discriminator loss: 4.0892 Generator loss: 0.9236
Epoch 1/2 Discriminator loss: 0.1595 Generator loss: 5.4472
Epoch 1/2 Discriminator loss: 1.6724 Generator loss: 2.9059
Epoch 1/2 Discriminator loss: 1.9217 Generator loss: 0.4889
Epoch 1/2 Discriminator loss: 1.8016 Generator loss: 10.5750
Epoch 1/2 Discriminator loss: 0.5429 Generator loss: 2.3942
Epoch 1/2 Discriminator loss: 2.3604 Generator loss: 2.2385
Epoch 1/2 Discriminator loss: 0.8042 Generator loss: 6.0856
Epoch 1/2 Discriminator loss: 1.3968 Generator loss: 2.6482
Epoch 1/2 Discriminator loss: 1.5982 Generator loss: 2.2070
Epoch 1/2 Discriminator loss: 1.5092 Generator loss: 1.7378
Epoch 1/2 Discriminator loss: 0.8052 Generator loss: 2.9875
Epoch 1/2 Discriminator loss: 0.5400 Generator loss: 2.1810
Epoch 1/2 Discriminator loss: 0.4807 Generator loss: 2.5932
Epoch 1/2 Discriminator loss: 0.6519 Generator loss: 2.8995
Epoch 1/2 Discriminator loss: 4.0717 Generator loss: 0.1662
Epoch 1/2 Discriminator loss: 6.3190 Generator loss: 10.3012
Epoch 1/2 Discriminator loss: 1.2158 Generator loss: 0.9179

Epoch 1/2 Discriminator loss: 2.0246 Generator loss: 0.5189
Epoch 1/2 Discriminator loss: 2.2147 Generator loss: 5.0791
Epoch 1/2 Discriminator loss: 3.0459 Generator loss: 0.2058
Epoch 1/2 Discriminator loss: 2.2342 Generator loss: 5.8493
Epoch 1/2 Discriminator loss: 1.0123 Generator loss: 2.3763
Epoch 1/2 Discriminator loss: 1.4389 Generator loss: 1.6461
Epoch 1/2 Discriminator loss: 1.2277 Generator loss: 4.4066
Epoch 1/2 Discriminator loss: 2.4671 Generator loss: 0.4705
Epoch 1/2 Discriminator loss: 2.3177 Generator loss: 2.1541
Epoch 1/2 Discriminator loss: 1.4624 Generator loss: 0.9856
Epoch 1/2 Discriminator loss: 3.1602 Generator loss: 0.2068
Epoch 1/2 Discriminator loss: 2.3706 Generator loss: 8.1830
Epoch 1/2 Discriminator loss: 0.5193 Generator loss: 4.7862
Epoch 1/2 Discriminator loss: 2.1419 Generator loss: 0.4233
Epoch 1/2 Discriminator loss: 1.3492 Generator loss: 1.2947
Epoch 1/2 Discriminator loss: 3.0735 Generator loss: 0.5674
Epoch 1/2 Discriminator loss: 2.8521 Generator loss: 1.4933
Epoch 1/2 Discriminator loss: 2.4502 Generator loss: 0.4756
Epoch 1/2 Discriminator loss: 2.3372 Generator loss: 0.7709
Epoch 1/2 Discriminator loss: 2.5739 Generator loss: 0.7713

Epoch 1/2 Discriminator loss: 1.5830 Generator loss: 2.4182
Epoch 1/2 Discriminator loss: 0.8396 Generator loss: 1.3273
Epoch 1/2 Discriminator loss: 0.8190 Generator loss: 1.3856
Epoch 1/2 Discriminator loss: 1.3070 Generator loss: 0.9180
Epoch 1/2 Discriminator loss: 1.4863 Generator loss: 1.2462
Epoch 1/2 Discriminator loss: 2.0782 Generator loss: 0.7163
Epoch 1/2 Discriminator loss: 2.3383 Generator loss: 0.5497
Epoch 1/2 Discriminator loss: 2.1272 Generator loss: 1.0293
Epoch 1/2 Discriminator loss: 1.5756 Generator loss: 1.2953
Epoch 1/2 Discriminator loss: 1.0672 Generator loss: 1.1706
Epoch 1/2 Discriminator loss: 1.0290 Generator loss: 1.3580
Epoch 1/2 Discriminator loss: 0.9603 Generator loss: 1.6913
Epoch 1/2 Discriminator loss: 0.9493 Generator loss: 1.9463
Epoch 1/2 Discriminator loss: 0.8733 Generator loss: 1.5617
Epoch 1/2 Discriminator loss: 1.1289 Generator loss: 1.1477
Epoch 1/2 Discriminator loss: 1.1127 Generator loss: 1.5715
Epoch 1/2 Discriminator loss: 1.2803 Generator loss: 1.0267
Epoch 1/2 Discriminator loss: 1.2626 Generator loss: 1.2043
Epoch 1/2 Discriminator loss: 1.2843 Generator loss: 1.2322
Epoch 1/2 Discriminator loss: 1.5654 Generator loss: 0.8654

Epoch 1/2 Discriminator loss: 1.7417 Generator loss: 1.4343
Epoch 1/2 Discriminator loss: 1.7298 Generator loss: 0.7891
Epoch 1/2 Discriminator loss: 1.3896 Generator loss: 1.0237
Epoch 1/2 Discriminator loss: 1.2844 Generator loss: 1.4120
Epoch 1/2 Discriminator loss: 1.1839 Generator loss: 1.1181
Epoch 1/2 Discriminator loss: 0.9225 Generator loss: 1.0853
Epoch 1/2 Discriminator loss: 1.1083 Generator loss: 1.4313
Epoch 1/2 Discriminator loss: 1.3122 Generator loss: 1.0522
Epoch 1/2 Discriminator loss: 1.4596 Generator loss: 1.0183
Epoch 1/2 Discriminator loss: 1.6910 Generator loss: 0.9127
Epoch 1/2 Discriminator loss: 1.9352 Generator loss: 0.5822
Epoch 1/2 Discriminator loss: 1.6599 Generator loss: 0.9306
Epoch 1/2 Discriminator loss: 1.4136 Generator loss: 1.1117
Epoch 1/2 Discriminator loss: 1.2210 Generator loss: 1.0785
Epoch 1/2 Discriminator loss: 1.1855 Generator loss: 1.1589
Epoch 1/2 Discriminator loss: 1.0622 Generator loss: 1.4053
Epoch 1/2 Discriminator loss: 1.2091 Generator loss: 0.8771
Epoch 1/2 Discriminator loss: 1.1616 Generator loss: 1.4110
Epoch 1/2 Discriminator loss: 1.3725 Generator loss: 0.9773
Epoch 1/2 Discriminator loss: 1.2037 Generator loss: 1.4861

Epoch 1/2 Discriminator loss: 1.4595 Generator loss: 0.8326
Epoch 1/2 Discriminator loss: 1.1280 Generator loss: 1.3149
Epoch 1/2 Discriminator loss: 1.3063 Generator loss: 0.8493
Epoch 1/2 Discriminator loss: 1.3208 Generator loss: 0.9522
Epoch 1/2 Discriminator loss: 1.3501 Generator loss: 1.1809
Epoch 1/2 Discriminator loss: 1.7081 Generator loss: 0.5937
Epoch 1/2 Discriminator loss: 1.6197 Generator loss: 1.0076
Epoch 1/2 Discriminator loss: 1.5133 Generator loss: 0.8211
Epoch 1/2 Discriminator loss: 1.1820 Generator loss: 1.7606
Epoch 1/2 Discriminator loss: 1.0942 Generator loss: 0.9895
Epoch 1/2 Discriminator loss: 0.9109 Generator loss: 1.4454
Epoch 1/2 Discriminator loss: 1.1610 Generator loss: 1.1056
Epoch 1/2 Discriminator loss: 1.6195 Generator loss: 0.6451
Epoch 1/2 Discriminator loss: 2.0098 Generator loss: 0.8382
Epoch 1/2 Discriminator loss: 1.7901 Generator loss: 0.5377
Epoch 1/2 Discriminator loss: 1.4218 Generator loss: 1.3532
Epoch 1/2 Discriminator loss: 1.6505 Generator loss: 0.5672
Epoch 1/2 Discriminator loss: 1.1516 Generator loss: 1.4015
Epoch 1/2 Discriminator loss: 1.2445 Generator loss: 1.5426
Epoch 1/2 Discriminator loss: 1.6369 Generator loss: 0.5188

Epoch 1/2 Discriminator loss: 1.6114 Generator loss: 1.4818
Epoch 1/2 Discriminator loss: 1.7663 Generator loss: 0.5850
Epoch 1/2 Discriminator loss: 1.6108 Generator loss: 1.2980
Epoch 1/2 Discriminator loss: 1.3391 Generator loss: 0.6566
Epoch 1/2 Discriminator loss: 1.2086 Generator loss: 1.2022
Epoch 1/2 Discriminator loss: 1.2790 Generator loss: 0.7589
Epoch 1/2 Discriminator loss: 1.3082 Generator loss: 1.1262
Epoch 1/2 Discriminator loss: 1.4288 Generator loss: 0.9839
Epoch 1/2 Discriminator loss: 1.3517 Generator loss: 1.0240
Epoch 1/2 Discriminator loss: 1.4186 Generator loss: 1.2419
Epoch 1/2 Discriminator loss: 1.2307 Generator loss: 0.9995
Epoch 1/2 Discriminator loss: 1.2529 Generator loss: 1.0796
Epoch 1/2 Discriminator loss: 1.2301 Generator loss: 1.0574
Epoch 1/2 Discriminator loss: 1.7384 Generator loss: 0.6881
Epoch 1/2 Discriminator loss: 2.0104 Generator loss: 0.8236
Epoch 1/2 Discriminator loss: 1.6319 Generator loss: 0.9786
Epoch 1/2 Discriminator loss: 1.4991 Generator loss: 0.7678
Epoch 1/2 Discriminator loss: 1.1550 Generator loss: 1.2666
Epoch 1/2 Discriminator loss: 1.4403 Generator loss: 0.9662
Epoch 1/2 Discriminator loss: 1.7362 Generator loss: 0.8426

Epoch 1/2 Discriminator loss: 1.5959 Generator loss: 0.8019
Epoch 1/2 Discriminator loss: 1.4225 Generator loss: 0.9027
Epoch 1/2 Discriminator loss: 1.5350 Generator loss: 0.7002
Epoch 1/2 Discriminator loss: 1.6254 Generator loss: 0.8284
Epoch 1/2 Discriminator loss: 1.3775 Generator loss: 0.8257
Epoch 1/2 Discriminator loss: 1.4931 Generator loss: 1.1008
Epoch 1/2 Discriminator loss: 1.1051 Generator loss: 1.0400
Epoch 1/2 Discriminator loss: 1.3005 Generator loss: 0.9340
Epoch 1/2 Discriminator loss: 1.0911 Generator loss: 1.0901
Epoch 1/2 Discriminator loss: 1.2731 Generator loss: 1.0384
Epoch 1/2 Discriminator loss: 1.6505 Generator loss: 0.8545
Epoch 1/2 Discriminator loss: 1.7372 Generator loss: 0.6294
Epoch 1/2 Discriminator loss: 1.4518 Generator loss: 0.9216
Epoch 1/2 Discriminator loss: 1.3230 Generator loss: 1.0553
Epoch 1/2 Discriminator loss: 1.2141 Generator loss: 0.9356
Epoch 1/2 Discriminator loss: 1.1801 Generator loss: 1.0056
Epoch 1/2 Discriminator loss: 1.2636 Generator loss: 1.1818
Epoch 1/2 Discriminator loss: 1.0468 Generator loss: 1.0196
Epoch 1/2 Discriminator loss: 1.3525 Generator loss: 1.0288
Epoch 1/2 Discriminator loss: 1.3389 Generator loss: 1.1662

Epoch 1/2 Discriminator loss: 1.5532 Generator loss: 0.5725
Epoch 1/2 Discriminator loss: 1.3513 Generator loss: 1.3512
Epoch 1/2 Discriminator loss: 1.7545 Generator loss: 0.5038
Epoch 1/2 Discriminator loss: 1.5252 Generator loss: 0.7610
Epoch 1/2 Discriminator loss: 1.6478 Generator loss: 0.8065
Epoch 1/2 Discriminator loss: 1.5797 Generator loss: 0.6522
Epoch 1/2 Discriminator loss: 1.3811 Generator loss: 0.9620
Epoch 1/2 Discriminator loss: 1.1770 Generator loss: 1.0616
Epoch 1/2 Discriminator loss: 1.2485 Generator loss: 0.8211
Epoch 1/2 Discriminator loss: 1.0700 Generator loss: 1.0943
Epoch 1/2 Discriminator loss: 1.1467 Generator loss: 0.7832
Epoch 1/2 Discriminator loss: 1.4438 Generator loss: 0.9891
Epoch 1/2 Discriminator loss: 1.5595 Generator loss: 0.7487
Epoch 1/2 Discriminator loss: 1.6657 Generator loss: 0.6885
Epoch 1/2 Discriminator loss: 1.5244 Generator loss: 0.7952
Epoch 1/2 Discriminator loss: 1.2377 Generator loss: 1.3452
Epoch 1/2 Discriminator loss: 1.2003 Generator loss: 0.7560
Epoch 1/2 Discriminator loss: 1.2541 Generator loss: 1.0295
Epoch 1/2 Discriminator loss: 1.2121 Generator loss: 1.3703
Epoch 1/2 Discriminator loss: 1.4772 Generator loss: 0.5659

CelebA¶

Run your GANs on CelebA. It will take around 20 minutes on the average GPU to run one epoch. You can run the whole epoch or stop when it starts to generate realistic faces.

batch_size = 128
z_dim = 100
learning_rate = 0.0002
beta1 = 0.5


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
epochs = 1

celeba_dataset = helper.Dataset('celeba', glob(os.path.join(data_dir, 'img_align_celeba/*.jpg')))
with tf.Graph().as_default():
    train(epochs, batch_size, z_dim, learning_rate, beta1, celeba_dataset.get_batches,
          celeba_dataset.shape, celeba_dataset.image_mode)

Epoch 1/1 Discriminator loss: 1.8545 Generator loss: 0.3563

Epoch 1/1 Discriminator loss: 0.8124 Generator loss: 2.5240
Epoch 1/1 Discriminator loss: 0.6391 Generator loss: 2.0169
Epoch 1/1 Discriminator loss: 0.4142 Generator loss: 1.9636
Epoch 1/1 Discriminator loss: 0.3166 Generator loss: 2.9372
Epoch 1/1 Discriminator loss: 0.2691 Generator loss: 3.0827
Epoch 1/1 Discriminator loss: 0.2800 Generator loss: 2.7259
Epoch 1/1 Discriminator loss: 0.2579 Generator loss: 3.4499
Epoch 1/1 Discriminator loss: 0.1406 Generator loss: 3.7267
Epoch 1/1 Discriminator loss: 1.3459 Generator loss: 0.6425
Epoch 1/1 Discriminator loss: 1.3725 Generator loss: 9.2074
Epoch 1/1 Discriminator loss: 0.4733 Generator loss: 1.8617
Epoch 1/1 Discriminator loss: 0.1177 Generator loss: 4.5872
Epoch 1/1 Discriminator loss: 0.8277 Generator loss: 1.3502
Epoch 1/1 Discriminator loss: 0.2171 Generator loss: 8.9523
Epoch 1/1 Discriminator loss: 0.3239 Generator loss: 3.5464
Epoch 1/1 Discriminator loss: 2.0811 Generator loss: 0.3815
Epoch 1/1 Discriminator loss: 2.4499 Generator loss: 14.7877
Epoch 1/1 Discriminator loss: 0.1492 Generator loss: 6.2942
Epoch 1/1 Discriminator loss: 6.7153 Generator loss: 0.0115

Epoch 1/1 Discriminator loss: 0.9689 Generator loss: 6.8048
Epoch 1/1 Discriminator loss: 1.4254 Generator loss: 1.5970
Epoch 1/1 Discriminator loss: 1.6526 Generator loss: 1.3414
Epoch 1/1 Discriminator loss: 1.3599 Generator loss: 3.9643
Epoch 1/1 Discriminator loss: 1.8715 Generator loss: 0.6020
Epoch 1/1 Discriminator loss: 1.2324 Generator loss: 6.5415
Epoch 1/1 Discriminator loss: 1.5944 Generator loss: 0.8846
Epoch 1/1 Discriminator loss: 0.9125 Generator loss: 3.2050
Epoch 1/1 Discriminator loss: 0.8914 Generator loss: 1.6605
Epoch 1/1 Discriminator loss: 1.3678 Generator loss: 1.3989
Epoch 1/1 Discriminator loss: 1.7390 Generator loss: 1.7252
Epoch 1/1 Discriminator loss: 3.0580 Generator loss: 0.3999
Epoch 1/1 Discriminator loss: 3.2611 Generator loss: 4.5681
Epoch 1/1 Discriminator loss: 3.6680 Generator loss: 0.1362
Epoch 1/1 Discriminator loss: 1.7782 Generator loss: 2.7195
Epoch 1/1 Discriminator loss: 1.9068 Generator loss: 1.0944
Epoch 1/1 Discriminator loss: 1.4524 Generator loss: 1.7045
Epoch 1/1 Discriminator loss: 1.3590 Generator loss: 1.1997
Epoch 1/1 Discriminator loss: 1.4264 Generator loss: 1.2851
Epoch 1/1 Discriminator loss: 1.6510 Generator loss: 1.4887

Epoch 1/1 Discriminator loss: 2.1196 Generator loss: 0.7386
Epoch 1/1 Discriminator loss: 1.2341 Generator loss: 2.0225
Epoch 1/1 Discriminator loss: 1.8119 Generator loss: 0.8495
Epoch 1/1 Discriminator loss: 1.6030 Generator loss: 2.8253
Epoch 1/1 Discriminator loss: 1.7529 Generator loss: 0.6302
Epoch 1/1 Discriminator loss: 1.3393 Generator loss: 1.7225
Epoch 1/1 Discriminator loss: 1.3545 Generator loss: 1.1789
Epoch 1/1 Discriminator loss: 1.2090 Generator loss: 1.9438
Epoch 1/1 Discriminator loss: 1.2461 Generator loss: 0.9931
Epoch 1/1 Discriminator loss: 1.3076 Generator loss: 1.7878
Epoch 1/1 Discriminator loss: 1.4280 Generator loss: 1.0870
Epoch 1/1 Discriminator loss: 1.4076 Generator loss: 2.4307
Epoch 1/1 Discriminator loss: 1.4278 Generator loss: 0.9847
Epoch 1/1 Discriminator loss: 1.1439 Generator loss: 2.4285
Epoch 1/1 Discriminator loss: 1.7940 Generator loss: 0.4784
Epoch 1/1 Discriminator loss: 1.6226 Generator loss: 4.0878
Epoch 1/1 Discriminator loss: 1.4743 Generator loss: 0.7419
Epoch 1/1 Discriminator loss: 1.2309 Generator loss: 1.2494
Epoch 1/1 Discriminator loss: 1.3579 Generator loss: 2.6765
Epoch 1/1 Discriminator loss: 2.5649 Generator loss: 0.2452

Epoch 1/1 Discriminator loss: 1.6532 Generator loss: 3.1526
Epoch 1/1 Discriminator loss: 1.9189 Generator loss: 0.4433
Epoch 1/1 Discriminator loss: 1.5294 Generator loss: 2.2155
Epoch 1/1 Discriminator loss: 1.6093 Generator loss: 0.7358
Epoch 1/1 Discriminator loss: 0.8473 Generator loss: 1.8885
Epoch 1/1 Discriminator loss: 1.2265 Generator loss: 1.1328
Epoch 1/1 Discriminator loss: 1.4633 Generator loss: 1.0972
Epoch 1/1 Discriminator loss: 1.5584 Generator loss: 1.3921
Epoch 1/1 Discriminator loss: 1.5806 Generator loss: 0.6928
Epoch 1/1 Discriminator loss: 1.3728 Generator loss: 1.4378
Epoch 1/1 Discriminator loss: 1.3193 Generator loss: 1.1556
Epoch 1/1 Discriminator loss: 1.3725 Generator loss: 1.2479
Epoch 1/1 Discriminator loss: 1.3418 Generator loss: 1.1072
Epoch 1/1 Discriminator loss: 1.7589 Generator loss: 0.5310
Epoch 1/1 Discriminator loss: 1.4913 Generator loss: 1.9206
Epoch 1/1 Discriminator loss: 1.8834 Generator loss: 0.6590
Epoch 1/1 Discriminator loss: 1.7407 Generator loss: 1.1766
Epoch 1/1 Discriminator loss: 1.7211 Generator loss: 1.0689
Epoch 1/1 Discriminator loss: 1.6334 Generator loss: 0.6698
Epoch 1/1 Discriminator loss: 2.0583 Generator loss: 1.0895

Epoch 1/1 Discriminator loss: 1.5336 Generator loss: 1.0839
Epoch 1/1 Discriminator loss: 1.3164 Generator loss: 1.2817
Epoch 1/1 Discriminator loss: 1.3801 Generator loss: 1.3004
Epoch 1/1 Discriminator loss: 1.8407 Generator loss: 0.4734
Epoch 1/1 Discriminator loss: 1.7742 Generator loss: 2.6494
Epoch 1/1 Discriminator loss: 2.1239 Generator loss: 0.3428
Epoch 1/1 Discriminator loss: 1.6328 Generator loss: 2.3797
Epoch 1/1 Discriminator loss: 1.6879 Generator loss: 0.6629
Epoch 1/1 Discriminator loss: 1.1005 Generator loss: 1.3293
Epoch 1/1 Discriminator loss: 1.2366 Generator loss: 1.0524
Epoch 1/1 Discriminator loss: 1.4360 Generator loss: 1.3278
Epoch 1/1 Discriminator loss: 1.1087 Generator loss: 1.1499
Epoch 1/1 Discriminator loss: 1.3976 Generator loss: 0.9164
Epoch 1/1 Discriminator loss: 1.1421 Generator loss: 1.9185
Epoch 1/1 Discriminator loss: 1.1776 Generator loss: 1.1178
Epoch 1/1 Discriminator loss: 1.1850 Generator loss: 1.2296
Epoch 1/1 Discriminator loss: 1.3445 Generator loss: 0.9204
Epoch 1/1 Discriminator loss: 1.3402 Generator loss: 1.3736
Epoch 1/1 Discriminator loss: 1.2763 Generator loss: 0.8635
Epoch 1/1 Discriminator loss: 1.1736 Generator loss: 1.6555

Epoch 1/1 Discriminator loss: 1.5666 Generator loss: 0.6891
Epoch 1/1 Discriminator loss: 1.1406 Generator loss: 1.3334
Epoch 1/1 Discriminator loss: 1.1858 Generator loss: 1.0899
Epoch 1/1 Discriminator loss: 1.0984 Generator loss: 1.2969
Epoch 1/1 Discriminator loss: 1.5534 Generator loss: 0.5105
Epoch 1/1 Discriminator loss: 1.7504 Generator loss: 1.7381
Epoch 1/1 Discriminator loss: 1.7519 Generator loss: 0.6672
Epoch 1/1 Discriminator loss: 1.4800 Generator loss: 0.9502
Epoch 1/1 Discriminator loss: 1.4506 Generator loss: 0.9622
Epoch 1/1 Discriminator loss: 1.5753 Generator loss: 0.9209
Epoch 1/1 Discriminator loss: 1.5643 Generator loss: 0.8322
Epoch 1/1 Discriminator loss: 1.6383 Generator loss: 1.0820
Epoch 1/1 Discriminator loss: 1.8625 Generator loss: 0.7957
Epoch 1/1 Discriminator loss: 1.9914 Generator loss: 0.7808
Epoch 1/1 Discriminator loss: 1.8213 Generator loss: 1.1007
Epoch 1/1 Discriminator loss: 1.5787 Generator loss: 1.0784
Epoch 1/1 Discriminator loss: 1.3682 Generator loss: 1.0454
Epoch 1/1 Discriminator loss: 1.2573 Generator loss: 1.0188
Epoch 1/1 Discriminator loss: 0.9662 Generator loss: 1.9410
Epoch 1/1 Discriminator loss: 1.1836 Generator loss: 0.8727

Epoch 1/1 Discriminator loss: 0.9063 Generator loss: 2.1529
Epoch 1/1 Discriminator loss: 0.9440 Generator loss: 1.3529
Epoch 1/1 Discriminator loss: 1.1896 Generator loss: 0.8078
Epoch 1/1 Discriminator loss: 1.0803 Generator loss: 1.4434
Epoch 1/1 Discriminator loss: 0.8706 Generator loss: 1.9546
Epoch 1/1 Discriminator loss: 1.1463 Generator loss: 0.7414
Epoch 1/1 Discriminator loss: 1.2491 Generator loss: 1.3778
Epoch 1/1 Discriminator loss: 1.0824 Generator loss: 1.2488
Epoch 1/1 Discriminator loss: 0.9400 Generator loss: 1.4211
Epoch 1/1 Discriminator loss: 0.9125 Generator loss: 1.4503
Epoch 1/1 Discriminator loss: 0.9697 Generator loss: 1.4487
Epoch 1/1 Discriminator loss: 1.1612 Generator loss: 0.8077
Epoch 1/1 Discriminator loss: 1.1030 Generator loss: 1.1834
Epoch 1/1 Discriminator loss: 0.9189 Generator loss: 1.2783
Epoch 1/1 Discriminator loss: 0.8710 Generator loss: 1.2271
Epoch 1/1 Discriminator loss: 0.8152 Generator loss: 1.6359
Epoch 1/1 Discriminator loss: 1.1780 Generator loss: 0.8949
Epoch 1/1 Discriminator loss: 0.9879 Generator loss: 1.4188
Epoch 1/1 Discriminator loss: 1.5235 Generator loss: 0.6387
Epoch 1/1 Discriminator loss: 1.5274 Generator loss: 2.4652

Epoch 1/1 Discriminator loss: 1.8631 Generator loss: 0.3679
Epoch 1/1 Discriminator loss: 1.4471 Generator loss: 1.4249
Epoch 1/1 Discriminator loss: 1.4697 Generator loss: 0.9529
Epoch 1/1 Discriminator loss: 1.4683 Generator loss: 0.7719
Epoch 1/1 Discriminator loss: 1.6335 Generator loss: 1.1950
Epoch 1/1 Discriminator loss: 1.3630 Generator loss: 0.8001
Epoch 1/1 Discriminator loss: 1.2356 Generator loss: 1.2192
Epoch 1/1 Discriminator loss: 1.3844 Generator loss: 0.8774
Epoch 1/1 Discriminator loss: 1.3027 Generator loss: 1.0249
Epoch 1/1 Discriminator loss: 1.3392 Generator loss: 1.6229
Epoch 1/1 Discriminator loss: 1.3534 Generator loss: 0.7783
Epoch 1/1 Discriminator loss: 1.6664 Generator loss: 0.6335
Epoch 1/1 Discriminator loss: 1.5333 Generator loss: 1.0977
Epoch 1/1 Discriminator loss: 1.6812 Generator loss: 0.7811
Epoch 1/1 Discriminator loss: 1.6339 Generator loss: 0.7048
Epoch 1/1 Discriminator loss: 1.2130 Generator loss: 1.2849
Epoch 1/1 Discriminator loss: 1.1897 Generator loss: 1.0825
Epoch 1/1 Discriminator loss: 1.3642 Generator loss: 0.9231
Epoch 1/1 Discriminator loss: 1.3728 Generator loss: 0.8471
Epoch 1/1 Discriminator loss: 1.4636 Generator loss: 0.9313

Epoch 1/1 Discriminator loss: 1.3128 Generator loss: 1.3142
Epoch 1/1 Discriminator loss: 1.4155 Generator loss: 0.9626
Epoch 1/1 Discriminator loss: 1.0855 Generator loss: 1.3992
Epoch 1/1 Discriminator loss: 1.3647 Generator loss: 0.6979
Epoch 1/1 Discriminator loss: 1.1737 Generator loss: 0.9885
Epoch 1/1 Discriminator loss: 1.3987 Generator loss: 0.8672
Epoch 1/1 Discriminator loss: 1.6362 Generator loss: 0.6670
Epoch 1/1 Discriminator loss: 1.4038 Generator loss: 0.7413
Epoch 1/1 Discriminator loss: 1.0789 Generator loss: 1.1381
Epoch 1/1 Discriminator loss: 1.1367 Generator loss: 1.0041
Epoch 1/1 Discriminator loss: 1.2700 Generator loss: 0.7890
Epoch 1/1 Discriminator loss: 1.4250 Generator loss: 0.7972
Epoch 1/1 Discriminator loss: 1.2528 Generator loss: 0.8810
Epoch 1/1 Discriminator loss: 1.2663 Generator loss: 1.0137
Epoch 1/1 Discriminator loss: 1.3626 Generator loss: 0.8556
Epoch 1/1 Discriminator loss: 1.7199 Generator loss: 0.5725
Epoch 1/1 Discriminator loss: 1.9309 Generator loss: 1.0765
Epoch 1/1 Discriminator loss: 1.9533 Generator loss: 0.9559
Epoch 1/1 Discriminator loss: 1.4777 Generator loss: 1.2976
Epoch 1/1 Discriminator loss: 1.5786 Generator loss: 0.5456

Submitting This Project¶

When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_face_generation.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "helper.py" and "problem_unittests.py" files in your submission.